1. Field of Endeavor
The present invention relates to data denoising and more particularly to parallel object-oriented data denoising.
2. State of Technology
U.S. Pat. No. 5,787,425 for an object-oriented data mining framework mechanism by Joseph Phillip Bigus, patented Jul. 28, 1998 provides the following description, “The development of the EDVAC computer system of 1948 is often cited as the beginning of the computer era. Since that time, computer systems have evolved into extremely sophisticated devices, capable of storing and processing vast amounts of data. As the amount of data stored on computer systems has increased, the ability to interpret and understand the information implicit in that data has diminished. In the past, data was stored in flat files, then hierarchical and network data based systems, and now in relational or object oriented databases. The primary method for analyzing that data has been to form well structured queries, for example using SQL (Structured Query Language), and then to perform simple aggregations or hypothesis testing against that data. Recently, a new technique called data mining has been developed, which allows a user to search large databases and to discover hidden patterns in that data. Data mining is thus the efficient discovery of valuable, non-obvious information from a large collection of data and centers on the automated discovery of new facts and underlying relationships in the data. The term “data mining” comes from the idea that the raw material is the business data, and the data mining algorithm is the excavator, shifting through the vast quantities of raw data looking for the valuable nuggets of business information. Because data can be stored in such a wide variety of formats and because the data values can have such a wide variety of meanings, data mining applications have in the past been written to perform specific data mining operations, and there has been little or no reuse of code between application programs. Thus, each data mining application is written from scratch, making the development process long and expensive. Although the nuggets of business information that a data mining application discovers can be quite valuable, they are of little use if they are expensive and untimely discovered. Returning to the mining analogy, even if gold is selling for $900 per ounce, nobody is interested in operating a gold mine if it takes two years and $901 per ounce to get it out of the ground.”
The journal article, “On Denoising Images Using Wavelet-based Statistical Techniques,” by Fodor, I. K. and C. Kamath, submitted to IEEE Transactions on Image Processing, March 2001 provides information about the state of the technology of denoising images. With sensors becoming ubiquitous and computers becoming more powerful, scientists are collecting and analyzing data at an ever increasing pace. This has resulted in several interesting problems in the analysis of data from areas as diverse as astronomy, medical imaging, and computer vision. In these fields, the data that is collected by sensors is often noisy, either as a result of the data acquisition process or due to natural phenomena such as atmospheric disturbances. Therefore, removing the noise from the data is an important problem that must be addressed before Applicants can analyze the data.
One approach to denoising data involves the thresholding of wavelet coefficients. Most methods in the literature have been designed for one-dimensional signals, but they can be extended to higher dimensional signals as well.
Various wavelet denoising techniques on two-dimensional data are compared and contrasted. Large-scale scientific data mining involves the analysis of massive datasets arising in scientific applications. As these data are frequently noisy, with the noise statistics varying from domain to domain, and sometimes from image to image, a software system was developed to enable experimentation with different options in wavelet denoising. The goal was three-fold. The first was to create a comprehensive object-oriented software library of wavelet denoising techniques to complement the extensive literature and existing software on the subject. While there are some packages such as WAVELAB that include denoising using wavelets, none provide a complete implementation of all the techniques proposed in the literature. Second, Applicants wanted to provide scientists, who are not experts in wavelet denoising, with a choice of techniques, so that they could select a combination appropriate for their data. Third, Applicants wanted to compare and contrast the various options in order to provide guidance and recommendations on their usage.
A section provides a brief introduction to denoising by thresholding of wavelet coefficients. Applicants explain the various options in denoising such as the choice of wavelet transforms, noise estimation techniques, threshold calculation methods, and threshold application schemes. A section contains a comprehensive evaluation of the various denoising combinations. Applicants compare the performance of the methods on test images with simulated noise and evaluate them with respect to the known noiseless images. Another Section compares the wavelet-based techniques to more traditional approaches to denoising based on spatial filters. The journal article, “On Denoising Images Using Wavelet-based Statistical Techniques,” by Fodor, I. K. and C. Kamath, submitted to IEEE Transactions on Image Processing, March 2001 is incorporated herein by this reference.